Analysis and Visualization of Classifier Performance: Comparison under Imprecise Class and Cost Distributions

Applications of inductive learning algorithms to realworld data mining problems have shown repeatedly that using accuracy to compare classifiers is not adequate because the underlying assumptions rarely hold. We present a method for the comparison of classifier performance that is robust to imprecise class distributions and misclassification costs. The ROC convex hull method combines techniques from ROC analysis, decision analysis and computational geometry, and --I--,LT.-.L11~. ..-.,I--.,-f --~-I---! aaapss r;nem 50 cne parr;iculars 01 analyzing iearned classifiers. The method is efficient and incremental, minimizes the management of classifier performance data, and allows for clear visual comparisons and sensitivity analyses.